1. Field of the Invention
This invention relates to artificial neural networks and neurocomputing systems for information processing, and more particularly, to photo stimulated imaging neural networks for providing self generating learning sets and associative memory and programmability. The self generating learning sets and associative memory and programmability are essential features for solving recognition and identification type problems associated with advanced robotics and control systems.
2. Discussion of the Prior Art
Standard computing or information processing systems at present are based on various implementations of the Von Neumann architecture. This type of information processing system relies on algorithms for implementing the various functions required for solving a particular problem. In order for the particular problem to be solved by the information processing system or computer, an individual; namely, a computer programmer must fully understand the particular problem and then be able to create an algorithm or series of algorithms capable of logically calculating a solution to the particular problem. Basically, the computer system is a "dumb" system that does not think for itself, but rather logically implements in a step-by-step process the instructions associated with a given algorithm to arrive at a solution. Although computers are "dumb", they are capable of many rote calculations per micro second and are therefore extremely useful in many applications. The inherent problem with this type of computing system is that the programmer must first theoretically solve or formulate functions to solve the specific problem if at all possible before the number crunching capabilities of the computer can be utilized. A second but associated problem is if the specific task or problem cannot be solved with a series of logical steps or arithmetic steps that will conclude with a solution to the particular problem. Even in cases where the complete function or task associated with a given problem is known it can still be impossible to develop an algorithm to implement the function in a certain case of problems because infinite examples exist. Such is the case, for example, for automobile autopilots, speech and handwriting translators, and identification systems for enemy aircraft or the like.
While computing capabilities and computers have advanced to solve a wide range of problems providing extraordinary speed, accuracy and convenience, the computer in many instances is still unable to solve the general class of problems associated with recognition and identification. In addition, standard computers do not have the ability to adapt to new and changing situations or learn on the fly. These types of problems limit the development of advanced robotics and control systems. However, in higher order biological systems recognition and identification functions are relatively simple tasks.
Neurocomputing is a new form of information processing which is emerging as a recognized alternative to the Von Neumann architecture. Neural computers are basically non-programmed adaptive information processing systems capable of developing associations between objects in response to their environment, as opposed to carrying out or implementing step-by-step procedures outlined by an algorithm. The neurocomputer generates its own rules governing the aforementioned association, refines these rules based on its own examples, and learns from its mistakes through this type of trial and error processing. Basically, the neurocomputer is trying to emulate the operation of a human brain in its most elemental form. The human brain is an extraordinary computing and information processing system which is particularly well suited for recognition type problems. While standard digital computers are far better suited to solving computational problems, the neurocomputers are best suited for solving complex pattern recognition problems inherent in target determination problems, continuous speech recognition, and handwriting analysis type problems.
Whereas the operation of the standard digital computer is well known, the operation of the human brain can only be characterized by gross observations and functions; therefore, the present state of neurocomputing has progressed to the point of utilizing basic circuits for implementing small associative memories and programmability. The associative memory and programmability in its simplest sense is one of being able to locate and retrieve data from various locations without having a specific address for the location or exactly what is to be located. In other words, the neurocomputer functions as the human brain does in attempting to recall from memory certain events by associating it with other events. This type of memory and programmability allows for more easily recognizing objects or patterns without exactly knowing what the object or pattern is.
The prior art contains a variety of papers and articles dealing with the neurocomputing area. The article "Artificial Neural Networks" by John J. Hopfield, IEEE Circuits and Devices Magazine 8755-3996/88/0900-0003 discusses the limitations associated with standard digital computers for recognition type problems and the advantages of neuro systems for solving the same type of problem. The article further discloses the basic Hopfield network which consists essentially of a set of amplifiers with sigmoid input/output characteristics, an input capacitance, an output resistance and a plurality of resistive connections. This electrical model of a neurobiological network is one of the basic building blocks for neuro computing networks. In the article "Computing with Neural Circuits" by John J. Hopfield et al., Science, Volume 233, Aug. 8, 1986, Hopfield et al. discusses recent work which is directed to how particular computations can be performed by utilizing an appropriate pattern of neuron amplifiers synaptically coupled in a simple dynamic model system. The basic premise behind the article is that a model of non-linear neuron amplifiers which have been organized into a network with symmetric connections has a natural capacity for solving optimization problems. The key to operation of the Hopfield network is high interconnectivity between the electrical components. The basic network consists essentially of the same elements as the aforementioned Hopfield article. In a third article "Neural Network and Physical Systems with Emergent Collective Computational Abilities", by John J. Hopfield, Proceedings National Academy of Science, Vol. 79, pp. 2554-2558, April 1982, Hopfield discusses a third aspect of neural networks in that the memory of these neural networks are retained as the stable states that the neural amplifiers of the neural network eventually settle into. Hopfield suggests that the link between simple electrical circuits and the complex computational properties of high order nervous systems may be the emergence of new computational capabilities from the collective behavior of a large number of simple processing elements such as the single neuron or neuron amplifier. The three Hopfield articles discuss and deal with a simple working model of the basic neural network utilizing standard electrical components.
Developments in artificial neural networks have offered new and useful concepts and in particular as presented by the three aforementioned articles, it has been shown that by high impedance coupling (soft coupling) with positive (enabling) feedback and negative (inhibitory) feedback in an array of amplifiers each having sigmoid transfer characteristics (remote cutoff and saturation) it is possible to mimic the biological neural system. A set of input conditions applied produces analog level transitions which are coupled into and between other amplifiers resulting in a dynamically changing set of levels which eventually settle into quasi stable states in either saturation or cutoff as effected by sigmoid responses. These states being unique to the given input data set can then by viewed as representing a learning set for that particular input. For a new input data set the artificial neural network is again initialized and run to exhibit a new learning set and so on.
The major difficulty and drawback to the aforementioned neural networks is that they can require a large number of discrete electrical inputs (dendrites) one for each amplifier (neuron). This makes it necessary to provide the required signal wire connections, processing and encoding of signals to satisfy the particular artificial neural network. In addition, the enabling and inhibitory functions which are controlled by the coupling resistors (synapses) must be changed for different input data sets requiring the resistors be set to new valves. Also, the input image data must be processed before entering the particular artificial neural network. The present invention avoids the drawbacks and difficulties presented in the prior art artificial neural networks by providing for the direct imaging and transfer of physical of sensor derived data and in the process, providing synaptic couplings which change and respond to the imaged data thereby avoiding the need to change valves.
The paper "An Optically Programmed Neural Network" by C. D. Kornfeld et al. presented at the IEEE SPIE conference in San Diego, Calif. on Jun. 18, 1988 discloses an approach as the name indicates for an optically programmed neural network which in combination with an external computer in a closed loop system calculates the network couplings by a convergent iterative process. Basically, the paper deals with the design, construction and operation of a hybrid electro-optic computer for use in neural networks. The system is designed around a photosensor array. The array in conjunction with its summing amplifiers constitutes a simple element performing the multiplication that is required by any neural network. The system has a computer incorporated into the loop between the input and output, and which is used to control and configure the array. The computer contains an optical program which implements a Hopfield style iterative memory. The present invention; however, is uniquely different in that it functions with a photo imaging and controlled coupling neural network to learn new images and detect them later by this learning set. The novel concept of the present invention is that it works directly with images that are impressed onto a photo receptor array to form unique weight, couplings and connectivity between neuron amplifiers to produce neural-like learning sets of stable amplifier states.